Image-data-based classification of vacuum seal packages

ABSTRACT

Vacuum seal packages can be classified based on image data. Training image data is received that includes image data about first vacuum seal packages. Labels associated with the first vacuum seal packages are received, where each of the labels includes a state of one of the first vacuum seal packages. A trained classification model is developed based on the training image data and the received labels. Image data representative of a second vacuum seal package is received. The image data is inputted into the trained classification model, where the trained classification model is configured to classify a state of the second vacuum seal package based on the image data. The state of the second vacuum seal package is received from the trained classification model.

BACKGROUND

The present disclosure is in the technical field of classification ofvacuum seal packages. More particularly, the present disclosure isdirected to training and using models to classify vacuum seal packagesbased on image data of the vacuum seal packages.

Vacuum seal packaging has been used for packaging various food products,such as poultry, meat, and cheese. In some cases, food products areplaced in heat sealable plastic bags. The air is then evacuated frominside the bag through a bag opening to collapse the bag around thecontained food product and the bag opening is heat sealed to fullyenclose the food product within the bag in a generally air-freeenvironment. In certain implementations, the bag is a heat shrinkablebag and the bagged product is advanced through a hot water or hot airshrink tunnel to cause the bag to shrink around the food product. Inother cases, food products are placed on trays with sealable film placedover the tray. The air is then evacuated from the tray through a tray orfilm opening to collapse the film over the food product in the tray andthe opening is sealed to fully enclose the food product within the trayin a generally air-free environment.

Vacuum seal packaging is effective at extending the life of foodproducts because of the generally air-free environment and because foodproduct is sealed from outside conditions. However, when the vacuum sealpackaging is defective, the benefits of vacuum seal packaging are lost.Under some circumstances, the food product inside of a defective vacuumseal package is exposed to conditions that allow the food product tospoil. In some cases, the food product then needs to be thrown out,resulting in costly waste. In other cases, the food product may beconsumed, resulting in discomfort or illness to the consumer. Thus, itis advantageous to ensure that the vacuum seal packages in which foodproducts are transported or sold are non-defective.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features ofthe claimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

In one embodiment, a system includes a transportation system, an imagesensor system, and one or more computing devices. The transportationsystem is configured to transport vacuum seal packages. Each of thevacuum seal packages includes a food product. The image sensor systemincludes an image data capture system. The image data capture system isarranged to capture image data of individual vacuum seal packages as thevacuum seal packages are transported by the transportation system. Theone or more computing devices are communicatively coupled to the imagesensor system and configured to receive the image data from the imagesensor system. The one or more computing devices include instructionsthat, in response to execution of the instructions by the one or morecomputing devices, cause the one or more computing devices to classify astate of one or more of the vacuum seal packages based on the image datausing a trained classification model and output the state of the one ormore of the vacuum seal packages after classification of the state ofthe one or more of the vacuum seal packages.

In one example, the trained classification model includes adecision-making process configured to receive an input that includes theimage data and to output an output that includes the state of the one ormore of the vacuum seal packages. In another example, thedecision-making process is a multilayer neural network, where themultilayer neural network includes an input layer includes the input, anoutput layer including the output, and at least one hidden layer betweenthe input layer and the output layer. In another example, the imagesensor system further includes a presence detector system configured todetect one of the vacuum seal packages on the transport system.

In another example, the image sensor system further includes acontroller, the controller is configured to receive a signal from thepresence detector system indicating the detected one of the vacuum sealpackages, and the controller is further configured to control a timingof the image sensor system during at least a portion of a time that theimage sensor system obtains the image data of the detected one of thevacuum seal packages. In another example, the transportation systemincludes a conveyor belt, and the controller is further configured tocontrol the timing of the image sensor system based in part on a speedof the conveyor belt. In another example, the classified state of theone or more of the vacuum seal packages includes an indication ofwhether the one or more of the vacuum seal packages is defective, isnon-defective, or has a particular defect. In another example, theclassified state of the one or more of the vacuum seal packages furtherincludes a degree of certainty as to whether the vacuum seal package isdefective, is non-defective, or has a particular defect. In anotherexample, the one or more computing devices are configured to output thestate of the one or more of the vacuum seal packages via by at least oneof providing an indication of the state to a user interface outputdevice, communicating the state via a communication interface to anexternal device, or storing the state in a local database.

In another embodiment, computer-readable medium have instructionsembodied thereon. The instructions include instructions that, inresponse to execution by one or more computing devices, cause the one ormore computing devices to perform a method. The method includesreceiving training image data, where the training image data includingimage data about a plurality of first vacuum seal packages. The methodfurther includes receiving labels associated with the plurality of firstvacuum seal packages, where each of the labels includes a state of oneof the plurality of first vacuum seal packages. The method furtherincludes developing a trained classification model based on the trainingimage data and the received labels and receiving image datarepresentative of a second vacuum seal package. The method furtherincludes inputting the image data into the trained classification model,where the trained classification model is configured to classify a stateof the second vacuum seal package based on the image data, and receivingthe state of the second vacuum seal package from the trainedclassification model.

In one example, the state of the second vacuum seal package includes anindication of whether the second vacuum seal package is defective, isnon-defective, or has a particular defect. In another example, the stateof the second vacuum seal package further includes a degree of certaintyas to whether the second vacuum seal package is defective, isnon-defective, or has a particular defect. In another example, theinstructions further include instructions that, in response to executionby the one or more computing devices, further cause the one or morecomputing devices to determine, based on the degree of certainty,whether a confidence level of the state of the second vacuum sealpackage is low and, in response to determining that the confidence levelof the state of the second vacuum seal package is low, to flag thesecond vacuum seal package for manual classification. In anotherexample, the instructions further include instructions that, in responseto execution by the one or more computing devices, further cause the oneor more computing devices to receive a user input of a manualclassification of the second vacuum seal package and further develop thetrained classification model based on the image data and the manualclassification of the second vacuum seal package.

In another example, the trained classification model includes adetection decision-making process and a classification decision-makingprocess. In another example, the detection decision-making process isconfigured to process the image data to produce processed image data. Inanother example, the detection decision-making process is configured toprocess the image data to produce processed image data at least bycropping an image in the image data so that the second vacuum sealpackage remains in the cropped image. In another example, the detectiondecision-making process is further configured to detect a presence ofthe second vacuum seal package in the image data. In another example,the classification decision-making process is configured to classify thestate of the second vacuum seal package based on the processed imagedata. In another example, the instruction that cause the one or morecomputing devices to develop a trained classification model includeinstructions that, in response to execution by the one or more computingdevices, cause the one or more computing devices to train theclassification model for a plurality of learning parameters anddetermine one or more model parameters based on the plurality oflearning parameters. In another example, the instruction that cause theone or more computing devices to develop a trained classification modelfurther include instructions that, in response to execution by the oneor more computing devices, cause the one or more computing devices tocreate the trained classification model based on the one or more modelparameters. In another example, the image data representative of thesecond vacuum seal package includes a plurality of forms of image data.In another example, the plurality of forms of image data includes atleast two images of the second vacuum seal package, and the trainedclassification model is configured to classify the state of the secondvacuum seal package based on the image data in part by separatelyclassifying a state of each of the at least two images of the secondvacuum seal package.

BRIEF DESCRIPTION OF THE DRAWING

The foregoing aspects and many of the attendant advantages of thedisclosed subject matter will become more readily appreciated as thesame become better understood by reference to the following detaileddescription, when taken in conjunction with the accompanying drawings,wherein:

FIGS. 1A and 1B depict top and side views of a system for classifyingthe state of vacuum seal packages, in accordance with the embodimentsdescribed herein;

FIGS. 2A and 2B depict top and side views of the system shown in FIGS.1A and 1B with an example of classifying the state of another type ofvacuum seal packages, in accordance with the embodiments describedherein;

FIG. 3 depicts a schematic diagram of an embodiment of an imageclassification system for classifying vacuum seal packages based onimage data of the vacuum seal packages, in accordance with theembodiments described herein;

FIG. 4A depicts an embodiment of a method of developing a trained imageclassification model, in accordance with the embodiments describedherein;

FIG. 4B depicts an example of a neural network that is a multilayerneural network, in accordance with the embodiments described herein;

FIG. 5 depicts an embodiment of a method of using a trained imageclassification model to classify a state of a vacuum seal package, inaccordance with the embodiments described herein;

FIG. 6 depicts an embodiment of a method of developing a trained imageclassification model based on a number of parameters, in accordance withthe embodiments described herein;

FIG. 7 depicts an embodiment of a method for an image classificationsystem to both train a model to classify states of vacuum seal packagesand apply the trained model to classify states of vacuum seal packages,in accordance with the embodiments described herein;

FIG. 8 depicts an embodiment of a method of classifying a state of avacuum seal package, in accordance with the embodiments describedherein;

FIG. 9 depicts an example embodiment of a system that may be used toimplement some or all of the embodiments described herein; and

FIG. 10 depicts a block diagram of an embodiment of a computing device,in accordance with the embodiments described herein.

DETAILED DESCRIPTION

To ensure that the vacuum seal packages in which food products aretransported or sold are non-defective, the vacuum seal packages can beinspected and labelled as defective or non-defective. However, manualinspection and labelling can be time-consuming and very costly. Inaddition, manual inspection and labelling of vacuum seal package isprone to human error. In some instances, inspectors who do not havesufficient training or experience can frequently falsely identifydefective packages as non-defective, leading to defective packages beingused to transport and sold, and falsely identify non-defective packagesas defective, leading to non-defective packages and food product beingwasted.

One solution may be to automatically label vacuum seal packages asdefective or non-defective, such as by a computer labelling the vacuumseal packages. Past attempts at automatic labelling including evaluatingimages of the vacuum seal packages for discontinuities in the vacuumseal packages, such as holes in the film, lack of sealing at film seams,and the like. However, these attempts had limited success because of theintricacies of looking for such discontinuities in images. In oneexample, a hole in the film may be as small as a pin head that is noteasily detectible in image of a vacuum seal package. In another example,the film used in a vacuum seal package may be transparent, which mayalso be difficult to detect in an image of a vacuum seal package. Inaddition, vacuum seal packages typically have physical features thatmake it difficult for an image to show the entire surface area of thefilm, such as folds and creases in the film, contours around the foodproduct, and the like.

The present disclosure describes embodiments of systems and methods ofclassifying vacuum seal packages based on image data using trainedmodels. A classification model can be trained to classify a vacuum sealpackage based on image data of the vacuum seal package. To train thetrained model, training image data is captured of a number of vacuumseal packages (e.g., hundreds of vacuum seal packages, thousands ofvacuum seal packages, or more). The training image data is manuallylabelled to classify a state (e.g., defective or non-defective) of thevacuum seal packages in the training image data. The labelled trainingimage data is used to develop the trained model to include adecision-making process (e.g., a decision tree, a neural network, etc.)that is optimized to classify the vacuum seal packages in the trainingimage data. After the model is trained, new image data of a vacuum sealpackage is provided to the trained model and the trained modelclassifies a state of the vacuum seal package represented in the newimage data. While the trained model does not necessarily “look” fordiscontinuities in the image data, the trained model can be much moreaccurate that manual classification and other forms of automaticclassification. Examples and variations of these embodiments and otherembodiments of training and using trained models are described herein.

Depicted in FIGS. 1A and 1B are top and side views of a system 100 forclassifying the state of vacuum seal packages. The system 100 includes atransportation system 102 configured to transport vacuum seal packages104 ₁, 104 ₂, and 104 ₃ (collectively vacuum seal packages 104) in atransportation direction 106. In the depicted embodiment, thetransportation system 102 includes a conveyor belt 108 on which thevacuum seal packages 104 are located. In the depicted embodiment, only aportion of the transportation system 102 is depicted; additional vacuumseal packages may be located on portions of the transportation system102 that are not depicted in FIGS. 1A and 1B.

In the depicted embodiment, each of the vacuum seal packages 104includes a food product 110. The food product 110 is vacuum sealedbetween a backing film 112 and a cover film 114. In some embodiments,the backing and cover films 112 and 114 are made from the same types offilm. In other embodiments, the backing and cover films 112 and 114 aremade from different types of film. In some embodiments, the backing andcover films 112 and 114 are heat sealed together. In other embodiments,the backing and cover films 112 and 114 are adhered together. In someembodiments, the backing and cover films 112 and 114 include an easyopen feature that enables consumers to open the vacuum seal packages 104without the use of tools (e.g., without the use of scissors).

The system 100 includes an image sensor system 116 that is configured toobtain image data of the vacuum seal packages 104. In some embodiments,the image sensor system 116 is configured to obtain image data of thevacuum seal packages 104 as the vacuum seal packages 104 are transportedby the transportation system 102 in the transportation direction 106. Insome embodiments, the image data obtained by the image sensor system 116of the vacuum seal packages 104 includes one or more images, one or morevideos, or any combination thereof.

In the depicted embodiment, the image sensor system 116 includes animage data capture system 118. The image data capture system 118includes a camera 120 configured to obtain image data within a field122. In some embodiments, the camera 120 includes one or more of asemiconductor charge-coupled device (CCD), an active pixel sensor in acomplementary metal-oxide-semiconductor (CMOS) integrated circuit, anactive pixel sensor in N-type metal-oxide-semiconductor (NMOS, Live MOS)integrated circuit, a three-dimensional (3D) sensor, a line scanner, orany other digital image sensor, or any combination thereof. In thedepicted embodiment, the camera 120 is arranged so that the field 122 isdirected toward a portion of the transportation system 102. In theinstance depicted in FIGS. 1A and 1B, the vacuum seal package 104 ₂ islocated on the conveyor belt 108 within the field 122 of the camera 120.With the vacuum seal package 104 ₂ in that location, the camera 120 isconfigured to obtain one or more images of the vacuum seal package 104₂, one or more videos of the vacuum seal package 104 ₂, or a combinationof images and videos of the vacuum seal package 104 ₂.

In some embodiments, the image data capture system 118 also includes oneor more electromagnetic energy sources 124 configured to emitelectromagnetic energy into the field 122 of the camera 120. In someembodiments, the one or more electromagnetic energy sources 124 areconfigured to emit electromagnetic energy in one or more of an X-rayrange of wavelengths (i.e., electromagnetic energy having a wavelengthbetween about 0.001 nm and about 10 nm), an ultraviolet range ofwavelengths (i.e., electromagnetic energy having a wavelength betweenabout 10 nm and about 400 nm), a visible range of wavelengths (i.e.,electromagnetic energy having a wavelength between about 380 nm andabout 760 nm), or an infrared range of wavelengths (i.e.,electromagnetic energy having a wavelength between about 750 nm andabout 1 mm). In some embodiments, the range(s) of wavelengths of theelectromagnetic energy emitted by the electromagnetic energy sources 124is determined based on a desired characteristic of the image dataobtained by the camera 120.

In the depicted embodiment, the image sensor system 116 also includes apresence detector system 126. In the depicted embodiment, the presencedetector system 126 is a photoelectric sensor (e.g., a photo eye). Morespecifically, the depicted embodiment of the presence detector system126 is a through-beam photoelectric sensor that includes a transmitter128 and a detector 130. The transmitter 128 is configured to emitelectromagnetic energy (e.g., infrared electromagnetic energy, visibleelectromagnetic energy, etc.) toward the detector 130. The detector 130is configured to detect the electromagnetic energy emitted by thetransmitter 128. If the detector 130 fails to detect the electromagneticenergy, the detector 130 can generate a signal indicative of an objectpassing between the transmitter 128 and the detector 130. In otherembodiments, the presence detector system 126 may be a through-beamphotoelectric sensor that includes a transceiver in place of thedetector 130 and a reflector in place of the transmitter 128. Thetransceiver emits electromagnetic energy toward the reflector, whichreflect the electromagnetic energy back to the transceiver. When anybreak in the electromagnetic energy is detected by the transceiver, thetransceiver can generate a signal indicative of an object passingbetween the transceiver and the reflector. In other embodiments, thepresence detector system 126 may be a diffusing photoelectric sensorthat is located on only one side of the transportation system 102 and iscapable of detecting the presence of an object on the conveyor belt 108.

In the depicted embodiment, the presence detector system 126 iscommunicatively coupled to a controller 132. When the presence detectorsystem 126 detects the presence of an object on the transportationsystem 102, the presence detector system is configured to communicate asignal to the controller 132 indicative of the presence of the object.The controller 132 is communicatively coupled to the image data capturesystem 118. The controller 132 is configured to cause the image datacapture system 118 to obtain image data of one of the vacuum sealpackages 104. In the embodiment shown in FIGS. 1A and 1B, the controller132 is external to both the image data capture system 118 and thepresence detector system 126. In this case, the controller 132 may be acomputing device in communication with each of the image data capturesystem 118 and the presence detector system 126. In other embodiments,the controller 132 may be integrated with either the image data capturesystem 118 or the presence detector system 126. In some embodiments, thecontroller 132 is capable of controlling the timing of the image datacapture system 118 so that one of the vacuum seal packages 104 is in thefield 122 of the camera 120 when the image data capture system 118obtains the image data.

In one example, as the transportation system 102 continues to move thevacuum seal packages 104 in the transportation direction 106, thepresence detector system 126 will detect the presence of the vacuum sealpackage 104 ₁ as the vacuum seal package 104 ₁ is moved between thetransmitter 128 and the detector 130, and the detector 130 sends asignal to the controller 132 indicative of the presence of the vacuumseal package 104 ₁. As the vacuum seal package 104 ₁ continues to movein the transportation direction 106, the controller 132 causes the imagedata capture system 118 to obtain image data of the vacuum seal package104 ₁. In some embodiments, the controller 132 controls the timing ofthe image data capture system 118 so that the vacuum seal package 104 ₁is within the field 122 if the camera 120 during at least a portion ofthe time that the camera obtains the image data of the vacuum sealpackage 104 ₁.

In the depicted embodiment, the image sensor system 116 iscommunicatively coupled to a computing device 134 via a network 136. Insome embodiments, the computing device 134 can be a remote computingdevice. As used herein, the term “remote computing device” refers to acomputing device that is located sufficiently far from a location that auser at the location cannot interact directly with the remote computerdevice. In other embodiments, the computing device 134 can be a localcomputing device. As used herein, the term “local computing device”refers to a computing device that is located at a location such that auser at the location can interact directly with the local computerdevice. The computing device 134 may be any type of computing device,such as a server, a desktop computer, a laptop computer, a cellulartelephone, a tablet, and the like.

In some embodiments, the network 136 is a wired network, such as aEthernet local area network (LAN), a coaxial cable data communicationnetwork, an optical fiber network, a direct wired serial communicationconnection (e.g., USB), or any other type of wired communicationnetwork. In some embodiments, the network 136 is a wireless network,such as a WiFi network, a radio communication network, a cellular datacommunication network (e.g., 4G, LTE, etc.), a direct wirelesscommunication connection (e.g., Bluetooth, NFC, etc.), or any other typeof wireless communication network. In some embodiments, the network 136is a combination of wired and wireless networks. In some embodiments,the network 136 may be a private network (e.g., a private LAN), a publicnetwork (e.g., the internet), or a combination of private and/or publicnetworks.

In some embodiments, the image sensor system 116 is configured to sendimage data obtained of the vacuum seal packages to the computing device134 via the network 136. In the depicted embodiment, the image datacapture system 118 is configured to send the image data to the computingdevice 134 via the network 136. The computing device 134 is configuredto classify a state of each of the vacuum seal packages 104 based on theimage data of each of the vacuum seal packages 104 received from theimage sensor system 116. In some embodiments, the state of a vacuum sealpackage classified by the computing device 134 includes a determinationof whether the vacuum seal package is defective or non-defective. Insome embodiments, the state of a vacuum seal package classified by thecomputing device 134 includes either an indication that the vacuum sealpackage is non-defective or an indication of at least one of a number ofpossible defects of the vacuum seal package. In some embodiments, thestate of a vacuum seal package classified by the computing device 134includes (1) an indication of at least one of vacuum seal package beingdefective, the vacuum seal package being non-defective, or a defect ofthe vacuum seal package, and (2) an indication of a degree of certaintyas to the indication of at least one of the vacuum seal package beingdefective, the vacuum seal package being non-defective, or a defect ofthe vacuum seal package. Examples of how the computing device 134 mayclassify a state of the vacuum seal packages 104 based on image data arediscussed below.

In some embodiments described herein, the state of a vacuum seal packagecan include a determination of a particular defect of the vacuum sealpackage. Examples of possible defects of vacuum seal packages include:tears in a bag that makes up the vacuum seal package, a low vacuumcondition of the vacuum seal package, an incorrect product locatedinside the vacuum seal package, a leaking vacuum seal package (e.g.,that is caused and/or defected by wrinkling, pin holes, seal failures,contamination, etc.), an oversized vacuum seal package for the size ofthe product in the vacuum seal, an incorrect establishment number, amisplaced label on the vacuum seal package, defective printing on thevacuum seal package, a hanging tail on the vacuum seal package, leakingvacuum seal packages caused by edge tears, a foreign object in thevacuum seal package, incorrect styling on the vacuum seal package (wherestyling indicates a position and/or orientation of objects inside thevacuum seal package), or any other type of defect.

Depicted in FIGS. 2A and 2B are top and side views of the system 100 inan example of classifying the state of another type of vacuum sealpackages. In FIGS. 2A and 2B, the system 100 includes the transportationsystem 102 and the image sensor system 116. The transportation system102 is configured to transport vacuum seal packages 204 ₁, 204 ₂, and204 ₃ (collectively vacuum seal packages 204) on the conveyor belt 108in the transportation direction 106. In the depicted embodiment, each ofthe vacuum seal packages 204 includes a food product 210. The foodproduct 210 is vacuum sealed between a tray 212 and a cover film 214. Insome embodiments, the tray 212 is a rigid tray configured tosubstantially hold its form before and after the vacuum sealing. In someembodiments, the cover film 214 is heat sealed to the tray 212. In otherembodiments, the cover film 214 is adhered to the tray 212. In someembodiments, the cover film 214 includes an easy open feature thatenables consumers to open the vacuum seal packages 204 without the useof tools (e.g., without the use of scissors).

The system 100 also includes the image sensor system 116 that isconfigured to obtain image data of the vacuum seal packages 204. In oneexample, as the transportation system 102 moves the vacuum seal packages204 in the transportation direction 106, the presence detector system126 will detect the presence of the vacuum seal package 204 ₁ as thevacuum seal package 204 ₁ is moved between the transmitter 128 and thedetector 130, and the detector 130 sends a signal to the controller 132indicative of the presence of the vacuum seal package 204 ₁. As thevacuum seal package 204 ₁ continues to move in the transportationdirection 106, the controller 132 causes the image data capture system118 to obtain image data of the vacuum seal package 204 ₁. In someembodiments, the controller 132 controls the timing of the image datacapture system 118 so that the vacuum seal package 204 ₁ is within thefield 122 if the camera 120 during at least a portion of the time thatthe camera obtains the image data of the vacuum seal package 204 ₁.

In some embodiments, the controller 132 is configured to control thetiming of the image data capture system 118 based on an expected size orshape of the vacuum seal packages. For example, the controller 132 maytake into account a distance between the middle of the vacuum sealpackages 204 in the transportation direction 106 and a position on thevacuum seal packages 204 that will first be detected by the presencedetector system 126. This allows the image data capture system 118 tocause the image data capture system 118 to capture image data of thevacuum seal packages 204 when the vacuum seal packages 204 are withinthe field 122 of the camera 120. It will be noted that the controller132 may be adjusted when a different type of vacuum seal packaging istransported by the transportation system 102, such as when vacuum sealpackages 104 are transported by the transportation system 102 in placeof the vacuum seal packages 204 shown in FIGS. 2A and 2B. In otherembodiments, the controller 132 may take into account a size of thevacuum seal packages 204. For example, the controller 132 may estimate awidth of the vacuum seal packages 204 based on an amount of time thatthe presence of the vacuum seal packages 204 is detected by the presencedetector system 126. In some embodiments, the controller 132 may takeinto account other aspects of the system 100, such as a speed of theconveyor belt 108, a shutter speed of the camera 120, or any othercharacteristics of the system 100.

As mentioned above, the computing device 134 may classify a state of thevacuum seal packages, such as vacuum seal packages 104 and vacuum sealpackages 204, based on image data of the vacuum seal packages. Depictedin FIG. 3 is a schematic diagram of an embodiment of an imageclassification system 300 for classifying vacuum seal packages based onimage data of the vacuum seal packages. The image classification system300 includes an image sensor system 302 and a computing device 310. Inthe embodiments of the systems 100 and 200, the image sensor system 302can be the image sensor system 116 and the computing device 310 can bethe computing device 134.

The image sensor system 302 configured to provide the computing device310 with image data of the vacuum seal packages. The image sensor system302 includes an image data capture system 304 configured to capture theimage data (e.g., take a picture or take video) of the vacuum sealpackages. The image sensor system 302 also includes a presence detectorsystem 306 configured to detect a presence of individual vacuum sealpackages. For example, the presence detector system 306 may detect apresence of individual vacuum seal packages as the vacuum seal packagesare transported by a transportation system. The image sensor system 302also includes a controller 308 configured to control a timing of theimage data capture by the image data capture system 304 based on signalsfrom the presence detector system 306. In the embodiments of the systems100 and 200, the image data capture system 304, the presence detectorsystem 306, and the controller 308 may be the image data capture system118, the presence detector system 126, and the controller 132,respectively.

The computing device 310 includes a processing unit 312, such as acentral processing unit (CPU). The processing unit is communicativelycoupled to a communication bus 314. In the depicted embodiment, thecomputing device 310 also includes memory 316 configured to store dataat the direction of the processing unit 312. In the depicted embodiment,the computing device 310 also includes a trained image classificationmodel 318 configured to classify a vacuum seal package based on imagedata of the vacuum seal package. Embodiments of trained models andtraining models are discussed in greater detail below. In the depictedembodiment, the computing device 310 also includes a user interface 320that includes one or more devices that are capable of receiving inputsfrom a user into the computing device 310 and/or outputting outputs fromthe computing device 310. In the depicted embodiment, the computingdevice 310 also includes a communication interface 322 that is capableof communicating with external computing devices and/or networks. In thedepicted embodiment, the computing device 310 also includes a database324 that is local to the computing device 310. Each of the memory 316,the trained image classification model 318, the user interface 320, thecommunication interface 322, and the database 324 is communicativelycoupled to the communication bus 314 so that the processing unit 312,the memory 316, the trained image classification model 318, the userinterface 320, the communication interface 322, and the database 324 arecapable of communicating with each other.

As noted above, the image sensor system 302 is configured to provide thecomputing device 310 with image data of the vacuum seal packages. Theimage data from the image sensor system 302 to the computing device 310may be communicated via one or more wired connections (e.g., a serialcommunication connection), wireless connections (e.g., a WiFiconnection), or a combination of wired and wireless connections. Uponthe computing device 310 receiving image data for a vacuum seal packagefrom the image sensor system 302, the processing unit 312 may cause theimage data to be stored in the memory 316. The processing unit 312 maythen instruct the trained image classification model 318 to classify astate of the vacuum seal package based on the image data stored in thememory 316. In some embodiments, the classified state of the vacuum sealpackage by the trained image classification model 318 may include anindication that the vacuum seal package is defective, is non-defective,or has a particular defect and/or an indication of a degree of certaintyas to whether the vacuum seal package is defective, is non-defective, orhas a particular defect. The processing unit 312 may then cause theclassification from the trained image classification model 318 to bestored in the memory 316.

After the image data is classified, the processing unit 312 may beconfigured to output the classification of the vacuum seal package. Insome embodiments, the processing unit 312 may output the classificationof the vacuum seal package by one or more of outputting theclassification of the vacuum seal package to a user via the userinterface 320, communicating the classification of the vacuum sealpackage to an external device via the communications interface 322, orlocally storing the classification of the vacuum seal package in thedatabase 324. In some cases, outputting the classification includesoutputting the classification only. In other cases, outputting theclassification includes outputting, with the classification, anidentification of the vacuum seal package, the image data associatedwith the vacuum seal package, a processed version of the image dataassociated with the vacuum seal package, metadata associated with theimage data, or any other information about the vacuum seal packageand/or the classification of the image data. In some embodiments wherethe classification of the vacuum seal package to an external device viathe communications interface 322, the classification can be communicatedfrom the communications interface 322 to an external computing device(e.g., a “cloud”-based server) that is configured to collect data aboutoperations and to analyze the data to improve performance (sometimesreferred to as an “internet of things” (IoT) service or interface). Insome embodiments where the classification of the vacuum seal package toan external device via the communications interface 322, theclassification can be communicated from the communications interface 322to a portion of a transportation system (e.g., the transportation system102) to route the vacuum seal package based on the classification.

As noted above, the trained image classification model 318 may bedeveloped to classify image data of vacuum seal packages. Depicted inFIG. 4A is an embodiment of a method 400 of developing a trained imageclassification model. At block 402, training image data of vacuum sealpackages is obtained. In some embodiments, the training image dataincludes images and/or video of vacuum seal packages having a knownstate. In some embodiments, the image data capture system used to obtainthe training image data is the same as the image data capture systemthat will be used to obtain image data of vacuum seal packages ofunknown state after the trained image classification model is created.At block 404, the training image data is manually labelled with thestates of the vacuum seal packages in the training image data. Forexample, a user can manually input a state (e.g., the vacuum sealpackage is defective, is non-defective, or has a particular defect) foreach image and/or video of a vacuum seal package in the image data.Manually labelling the image data may include physically testing thevacuum seal packages to determine whether individual vacuum sealpackages are sealed or leak and then applying a label to the image databased on the results of the physical testing. In some embodiments, thenumber of vacuum seal packages represented in the training image data isin a range of tens of vacuum seal packages, hundreds of vacuum sealpackages, thousands of vacuum seal packages, or more. At these numbers,the manual labelling process of the training image data may be alabor-and time-intensive process. At block 406, the labelled trainingimage date is input into a training module. In some embodiments, thetraining model is a machine learning module, such as a “deep learning”module. Deep learning is a subset of machine learning that generatesmodels based on training data sets provided to it.

At block 408, the trained model is developed to classify vacuum sealpackages. In some embodiments, as the trained model is developed, one ormore learning algorithms are used to create the trained model based onthe labelled states of the vacuum seal packages in the training imagedata. In some embodiments, the trained model is created based on inputvectors which are indicative of a characteristic of the vacuum sealpackages. In one example, the input vector may be the “looseness” of thevacuum seal package, with looser vacuum seal packages defined asdefective. In one example, the looseness above a particular thresholdmay indicate a loss of vacuum within the vacuum seal package. In otherexamples, the input vectors may be colors in the visible spectrum,detection of an additive in a film of the vacuum seal package using anon-visible electromagnetic energy (e.g., ultraviolet, infrared), thepresence and numbers of film folds, or any other number of possibleinput vectors. The use of input vectors for training may help thetrained model identify defective vacuum seal packages withoutidentifying the underlying cause. For example, a vacuum seal package mayhave a small pinhole that would be difficult to detect using image datacaptured as the vacuum seal package is being moved on a transportationsystem. The use of the input vectors allows the trained model to detectthat the vacuum seal package is defective without the need to identifythe small pinhole itself. After the input vectors are modeled, a trainedmodel can be developed as a decision-making process based on a number ofthe input vectors. Examples of decision-making processes includedecision trees, neural networks, and the like. In some embodiments, thedecision-making process of the trained model is based on a determinationof an acceptable arrangement of the input vectors in the decision-makingprocess.

The result of the development of the trained model in block 408 is thetrained model depicted at block 410. The trained model can be usedduring normal operation (e.g., operation that is not used to train tothe trained model) to identify states of vacuum seal packages. In someembodiments, the trained model includes a neural network that has anumber of layers. Depicted in FIG. 4B is an example of a neural network420 that is a multilayer neural network. In the depicted embodiment, theneural network 420 includes a first layer 422 with three input nodes, asecond layer 424 with five hidden nodes, a third layer 426 with fourhidden nodes, a fourth layer 428 with four hidden nodes, and a fifthlayer 430 with one output node. The neural network 420 also includes afirst set of connections 432 between each pair of the three input nodesin the first layer and the five input nodes in the second layer 424, asecond set of connections 434 between each pair of the five input nodesin the second layer 424 and the four hidden nodes in the third layer426, a third set of connections 436 between each pair of the four hiddennodes in the third layer 426 and the four hidden nodes in the fourthlayer 428, and a fourth set of connections 438 between each pair of thefour hidden nodes in fourth layer 428 and the output node in the fifthlayer 430. In some embodiments, the input nodes represent inputs intothe trained models (e.g., image data, metadata associated with the imagedata, etc.), one or more of the hidden nodes (e.g., one of the layers ofhidden nodes) may represent one of the input vectors determined duringthe development of the model, and the output node represents thedetermined state of the vacuum seal package.

Depicted in FIG. 5 is an embodiment of a method 500 of using a trainedimage classification model to classify a state of a vacuum seal package.At block 502, image data of the vacuum seal package is acquired. Theimage data of the vacuum seal package may be obtained by an image datacapture system, such as an image data capture system in an image sensorsystem. In some embodiments, the image data of the vacuum seal packageis obtained while the vacuum seal package is being transported by atransport system.

At block 504, the image data of the vacuum seal package is input into atrained image classification model. The trained image classificationmodel may be operating on a computing device, such as a local computingdevice at the image data capture system or a remote computing devicefrom the local computing device. The trained image classification modelis configured to classify a state of the vacuum seal package based onthe image data. At block 506, a classification of a state of the vacuumseal package is received from the trained image classification model. Insome embodiments, the classified state includes an indication that avacuum seal package is defective, is non-defective, or has a particulardefect, and/or an indication of a degree of certainty as to whether thevacuum seal package is defective, is non-defective, or has a particulardefect. In some embodiments, the classified state is received by one ormore of displaying the classification on a user interface output device,communicating the classification via a communication interface to one ormore external devices, or storing the classification in a database. Insome embodiments, the received classification the vacuum seal packageincludes one or more of the classified state or the vacuum seal packageor a degree of certainty of the classified state of the classified stateor the vacuum seal package. In one specific example, the state iscommunicated to a routing system that is configured to route vacuum sealpackages on a transportation system based on their states, such asrouting defective packages to a location for repackaging and/or wastedisposal.

As noted above, the method 400 is used to obtain the trainedclassification model at block 410 and then the trained classificationmodel can be used in method 500 to classify vacuum seal packages. Insome embodiments, the training image data acquired at block 402 is imagedata of a particular type of vacuum seal packages and the image dataacquired at block 502 is image data of the same type of vacuum sealpackages. In one example, the training image data acquired at block 402is image data of the vacuum seal packages 104 and the image dataacquired at block 502 is image data of the same type of vacuum sealpackages as the vacuum seal packages 104. In some embodiments, thetraining image data acquired at block 402 is image data of a particulartype of vacuum seal packages and the image data acquired at block 502 isimage data of a different type of vacuum seal packages. In one example,the training image data acquired at block 402 is image data of thevacuum seal packages 104 and the image data acquired at block 502 isimage data of the vacuum seal packages 204. Even though the vacuum sealpackages 204 are a different type from the vacuum seal packages 104, thetrained classification model using the training image data from thevacuum seal packages 104 may be able to classify states of the vacuumseal packages 204 with sufficient accuracy.

Depicted in FIG. 6 is an embodiment of a method 600 of developing atrained image classification model. At block 602, training image data isacquired for a number of vacuum seal packages. At block 604, thetraining image data is manually labelled as defective or non-defective.The manual labelling of the training image data may be done by a userentering an indication of defective or non-defective for each of thevacuum seal packages represented in the training image data into a userinterface input device of a computing device.

At block 606, model information, training objectives, and constraintsare initialized. In some examples, model information includes a type ofmodel to be used, such as a neural network, a number of input vectors,and the like. In some examples, training objectives can include adesired or expected performance of the trained model, such as anaccuracy rate of greater than or equal to a predetermined rate (e.g.,greater than or equal to one or more of 90%, 95%, 96%, 97%, 98%, or99%). In some examples, constraints can include limitations of thetrained model, such as a minimum number of layers of a neural network, amaximum number of layers of a neural network, a minimum weighting ofinput vectors, a maximum weighting of input vectors, or any otherconstraints of a trained model. At block 608, the model can be trainedusing the model information and the model constraints. In someembodiments, the training image data is separated into two subsets—atraining subset and a validation subset—and the training of the model atblock 608 includes training the model using the training subset of theimage data.

At block 610, a determination is made whether the training objective ismet. In some embodiments, the determination at block 610 is made bycomparing the results of the trained model to the training objectiveinitialized at block 606. In some embodiments, where the training imagedata is separated into the training subset and the validation subset,the determination at block 610 includes testing the model trained atblock 608 using the validation subset of the image data. If, at block610, a determination is made that the training objective is not met,then the method 600 proceeds to block 612 where the training objectiveand/or the constraints are updated. After the training objective and/orthe constraints are updated at block 612, the method 600 returns toblock 608 where the model is trained using the updated trainingobjective and/or constraints. If, at block 610, a determination is madethat the training objective is met, then the method 600 proceeds toblock 614 where the trained model is stored. Storing the trained modelmay include storing the trained model in one or more memories in acomputing device (e.g., a local computing device, a remote computingdevice, etc.).

In some embodiments, an image classification system may be used both totrain a model to classify states of vacuum seal packages and to applythe trained model to classify states of vacuum seal packages. Depictedin FIG. 7 is an embodiment of a method 700 for an image classificationsystem to both train a model to classify states of vacuum seal packagesand apply the trained model to classify states of vacuum seal packages.In some embodiments, the image classification system includes an imagesensor system and a computing device (e.g., the image sensor system 302and the computing device 310 of the image classification system 300). Inthose embodiments, the model may operate on the computing device whilethe image sensor system obtains image data of vacuum seal packageseither for training or applying the model.

At block 702, the image classification system and the classificationmodel are initialized. In some embodiments, initialization of the imageclassification system includes initializing a computing device andinitializing an image sensor system, and initialization of theclassification model includes loading launching software that includesthe classification model on the computing system. At block 704, theimage data of a vacuum seal package is acquired. In some embodiments,the image sensor system acquires the image data of the vacuum sealpackage and provides the image data to the computing system. At block706, a determination is made whether the classification model is intraining mode. The determination may be made by the software operatingon the computing system that includes the classification model.

If, at block 706, a determination is made that the classification modelis in training mode, then the model passes to block 708, where adetermination is made if a state is available for the vacuum sealpackage. A state may be available for a vacuum seal package when a usermanually enters a state for the vacuum seal package into a computingdevice. If, at block 708, a determination is made that a state isavailable, then the method proceeds to block 710. At block 710, theclassification model is updated based on the image data and the statefor the vacuum seal package. Updating the classification model caninclude any of the methods described herein for training and/ordeveloping classification models. At this point, a vacuum seal packagestate (e.g., the manually-entered state) is available, as shown in block712. However, if, at block 706, a determination is made that theclassification model is not in training mode or if, at block 708, adetermination is made that a state is not available, then the methodproceeds to block 714.

At block 714, the classification model classifies a state of the vacuumseal package. In some embodiments, the state of a vacuum seal packageclassified by the classification model includes a determination ofwhether the vacuum seal package is defective, is non-defective, or has aparticular defect, and an indication of a degree of certainty as towhether the vacuum seal package is defective, is non-defective, or has aparticular defect. At block 716, a determination is made whether aconfidence level of the classified state is low. In some embodiments,the confidence level is a percentage representing the degree ofcertainty that the classified state of the vacuum seal package isaccurate and confidence level is low if the degree of certainty is belowa predetermined percentage of an acceptable degree of certainty. Forexample, if the acceptable degree of certainty is 90%, then theclassified state of the vacuum seal package is deemed to be low if thedegree of certainty of the classified state is below 90%. If, at block716, the confidence level is determined to not be low, then the vacuumseal package state has been determined, as shown at block 718. However,if at block 716, the confidence level is determined to be low, then themethod proceeds to block 720 where the vacuum seal package is set asidefor manual classification (e.g., classification by a user after visualinspection or physical testing).

If a state of the vacuum seal package is available, either at block 712or at block 718, then the method proceeds to block 722. At block 722,the state of the vacuum seal package is output. In some embodiments,outputting the state of the vacuum seal package includes one or more ofdisplaying the state of the vacuum seal package on a user interfaceoutput device, communicating the state of the vacuum seal package via acommunication interface to one or more external devices, or storing thestate of the vacuum seal package in a database. In some embodiments, thestate of the vacuum seal package includes one or more of an indicationof whether the vacuum seal package is defective, is non-defective, orhas a particular defect, or a degree of certainty of whether the vacuumseal package is defective, is non-defective, or has a particular defect.

Whether state of the vacuum seal package is output at block 722 or thevacuum seal package is held for manual classification at block 720, themethod 700 then proceeds to block 724. At block 724, a determination ismade whether another vacuum seal package is available. In someembodiments, the determination at block 724 can be based on whetheranother vacuum seal package is detected on a transportation system(e.g., whether the presence detector system 126 detects another vacuumseal package on the transportation system 102). In some embodiments, thedetermination at block 724 can be based on whether a user inputs anindication whether another vacuum seal package is available. If, atblock 724, a determination is made that another vacuum seal package isnot available, then, at block 726, the image data capture system and theclassification model are shut down. However, if, at block 724, adetermination is made that another vacuum seal package is available,then the method 700 loops back to block 704 where image data is acquiredof the next vacuum seal package and the method 700 proceeds from block704 as described above for the next vacuum seal package.

As discussed above, a trained model to classify states of vacuum sealpackages from image data may include one decision-making process, suchas a decision tree or a neural network. In some embodiments, a trainedmodel to classify states of vacuum seal packages from image data mayinclude more than one decision-making process. Depicted in FIG. 8 is anembodiment of a method 800 of classifying a state of a vacuum sealpackage. In the depicted embodiment, the method 800 is performed in partby an image sensor system 802, a detection decision-making process 804,a classification decision-making process 806, and an output device 808.At block 810, the image sensor system acquires image data of a vacuumseal package. In some embodiments, the image sensor system 802 mayacquire the image data as the vacuum seal package is being transportedby a transport system. After the image data is acquired at block 810,the image sensor system has image data 812 that can be communicated tothe detection decision-making process 804. In some embodiments, thedetection decision-making process 804 is a software-baseddecision-making process operating on one or more computing devices.

At block 814, the detection decision-making process 804 processes theimage data received from the image sensor system 802. In someembodiments, the processing of the image data at block 814 is performedby a trained model that has been trained to detect a region of interestassociated with a vacuum seal package in image data. In someembodiments, the processing of the image data at block 814 includes oneor more of cropping an image in the image data around a detected vacuumseal package in the image, selecting a frame or a subset of frames froma video in the image data, identifying irrelevant pixels from an imagein the image data and replacing the irrelevant pixels with the leastsignificant values of the image data. In some embodiments, theprocessing of the image data produces a single image having arectangular shape with the identified vacuum seal package substantiallycentered in the image and the pixels deemed to be irrelevant beingreplaced with the least significant values. In some embodiments, theprocessing of the image data can include masking a portion of an image,where areas of the image outside of a region of interest (e.g., outsideof a vacuum seal package) are replaced with low value data (e.g., thepixels are all changed to black) to reduce the amount of processing toclassify the state of the vacuum seal package and reduce the likelihoodof error when classifying the state of the vacuum seal package.

In one particular embodiment of processing image data, a custom boundaryis constructed around a representation of a vacuum seal package in theimage data. A bounding box encompassing the vacuum seal package is alsoconstructed in the custom boundary. The processing also includescropping the bounding box from the entire image data. One advantage ofcropping the image data based on the custom boundary is that the laterclassification of the state of the vacuum seal package may be limited toareas of interest without the need to inspect areas of the image datathat are not of interest. This may, in turn, increase the confidencelevel of classification and therefore overall accuracy of theclassification. In some embodiments, where the detection decision-makingprocess 804 is a multilayer neural network, creating the bounding boxaround the custom boundary simplifies compatibility requirements betweenthe image data and the first layer of the neural network. Additionally,cropping the image data results in a portion of the image data beingprocessed for classification, rather than the entire image data, whichreduces the processing time for classification. In some embodiments, thecustom boundary may help in generating a numerical value for one or moreof the area of the vacuum seal package, its centroid, or itsorientation.

At block 816, a determination is made whether the presence of a vacuumseal package is detected in the image data. In some embodiments, thedetermination made at block 816 is a part of the processing of the imagedata at block 816. In some embodiments, the determination of whethervacuum seal package is detected at block 816 is a separate process fromthe processing of the image data at block 816. If, at block 816, adetermination is made that the presence of a vacuum seal package is notdetected, then the method 800 proceeds to block 818 where the image datais discarded (e.g., deleted) and the method 800 ends. However, if, atblock 816, a determination is made that the presence of a vacuum sealpackage is detected, then the processed image data represented at block820 can be communicated to the classification decision-making process806. In some embodiments, the classification decision-making process 806is a software-based decision-making process operating on one or morecomputing devices, which may be the same as or different from the one ormore computing devices on which the detection decision-making process804 operates. In some embodiments, processing the image data at block814 to obtain the processed image data, as shown at block 820, prior toclassifying a state of the vacuum seal package represented in the dataincreases the accuracy of the later-performed classification by theclassification decision-making process 806.

At block 822, the classification decision-making process 806 classifiesthe processed image data received from the detection decision-makingprocess 804. In some embodiments, the classification of the image dataat block 822 is performed by a trained model that has been trained toclassify a state of vacuum seal packages represented in processed imagedata. In some embodiments, the classification of the state of the vacuumseal package represented in the processed image data at block 822includes a determination of whether the vacuum seal package isdefective, is non-defective, or has a particular defect. In someembodiments, the classification of the state of the vacuum seal packagerepresented in the processed image data at block 822 includes adetermination of whether the vacuum seal package is defective, isnon-defective, or has a particular defect, and an indication of a degreeof certainty as to whether the vacuum seal package is defective, isnon-defective, or has a particular defect.

At block 824, a determination is made whether a confidence level of theclassified state is low. In some embodiments, the confidence level is apercentage representing the degree of certainty that the classifiedstate of the vacuum seal package is accurate and the confidence level islow if the degree of certainty is below a predetermined percentage of anacceptable degree of certainty. For example, if the acceptable degree ofcertainty is 90%, then the classified state of the vacuum seal packageis deemed to be low if the degree of certainty of the classified stateis below 90%. If, at block 824, the confidence level is determined tonot be low, then the vacuum seal package state has been determined, asshown at block 826. However, if at block 824, the confidence level isdetermined to be low, then the method proceeds to block 828 where thevacuum seal package and/or the image data is flagged for manualclassification.

At block 830, a state of the vacuum seal package is manually classifiedoutside of the classification decision-making process. In someembodiments, the vacuum seal package is manually classified by a userafter visual inspection or physical testing of the vacuum seal package.At block 832, the user inputs the manually-classified state of thevacuum seal package to the classification decision-making process 806.At block 834, the classification decision-making process 806 is updated.In embodiments where the classification decision-making process 806 is atrained model, updating the classification decision-making process 806includes further training the trained model based on the manualclassification. After updating the classification decision-makingprocess 806, the method 700 returns to block 826 where the classifiedstate of the vacuum seal package is the manually-classified state of thevacuum seal package.

After the classified state of the vacuum seal package, as represented atblock 826, is classified or obtained by the classificationdecision-making process 806, the classification decision-making process806 sends the classified stated of the vacuum seal package to the outputdevice 808. In the embodiments where the classification decision-makingprocess 806 is software operating on one or more computing devices, theoutput device 808 can be a user interface output device. In someembodiments, the outputting the classified state of the vacuum sealpackage at block 836 includes one or more of outputting the classifiedstate of the vacuum seal package to a user via a user interface (e.g., amonitor, a touchscreen, etc.), communicating the classified state of thevacuum seal package to an external device via a communicationsinterface, or locally storing the classified state of the vacuum sealpackage in a database.

In any of the embodiments disclosed herein, the image data received forany one vacuum seal package may include multiple forms of image dataabout the same vacuum seal package. For example, image data about avacuum seal package may include two images in the visible light range ofthe same vacuum seal package. These multiple different forms of imagedata for the same vacuum seal package may be passed through a trainedmodel separately. If the trained model returns the same classified statefor the vacuum seal package using the two different forms of image data,then the confidence level of the classification for that vacuum sealpackage can be increased significantly. In one example, if the trainedmodel classified one of the images as having a vacuum seal package withan imperfect seal at a 98% confidence level and classified the otherimage as having a vacuum seal package with an imperfect seal at a 96%confidence level, then the confidence level that the vacuum seal packagehas an imperfect seal may be greater than 99%. In another example, ifthe trained model classified one of the images as having a non-defectivevacuum seal package at a 60% confidence level and classified the otherimage as having a non-defective vacuum seal package at an 70% confidencelevel, then the confidence level that the vacuum seal package isnon-defective may be 88%. Even though the confidence level using twoimages may be significantly higher than either of the images alone, thecombined confidence level from two images (e.g., 88%) may still be belowa predetermined percentage of an acceptable degree of certainty (e.g.,95%), which may cause the vacuum seal package to be flagged for manualclassification. It will be apparent that the number of multiple forms ofimage data is not limited to two, but could be any number of forms ofimage data.

FIG. 9 depicts an example embodiment of a system 910 that may be used toimplement some or all of the embodiments described herein. In thedepicted embodiment, the system 910 includes computing devices 920 ₁,920 ₂, 920 ₃, and 920 ₄ (collectively computing devices 920). In thedepicted embodiment, the computing device 920 ₁ is a tablet, thecomputing device 920 ₂ is a mobile phone, the computing device 920 ₃ isa desktop computer, and the computing device 920 ₄ is a laptop computer.In other embodiments, the computing devices 920 include one or more of adesktop computer, a mobile phone, a tablet, a phablet, a notebookcomputer, a laptop computer, a distributed system, a gaming console(e.g., Xbox, Play Station, Wii), a watch, a pair of glasses, a key fob,a radio frequency identification (RFID) tag, an ear piece, a scanner, atelevision, a dongle, a camera, a wristband, a wearable item, a kiosk,an input terminal, a server, a server network, a blade, a gateway, aswitch, a processing device, a processing entity, a set-top box, arelay, a router, a network access point, a base station, any otherdevice configured to perform the functions, operations, and/or processesdescribed herein, or any combination thereof.

The computing devices 920 are communicatively coupled to each other viaone or more networks 930 and 932. Each of the networks 930 and 932 mayinclude one or more wired or wireless networks (e.g., a 3G network, theInternet, an internal network, a proprietary network, a securednetwork). The computing devices 920 are capable of communicating witheach other and/or any other computing devices via one or more wired orwireless networks. While the particular system 910 in FIG. 9 depictsthat the computing devices 920 communicatively coupled via the network930 include four computing devices, any number of computing devices maybe communicatively coupled via the network 930.

In the depicted embodiment, the computing device 920 ₃ iscommunicatively coupled with a peripheral device 940 via the network932. In the depicted embodiment, the peripheral device 940 is a scanner,such as a barcode scanner, an optical scanner, a computer vision device,and the like. In some embodiments, the network 932 is a wired network(e.g., a direct wired connection between the peripheral device 940 andthe computing device 920 ₃), a wireless network (e.g., a Bluetoothconnection or a WiFi connection), or a combination of wired and wirelessnetworks (e.g., a Bluetooth connection between the peripheral device 940and a cradle of the peripheral device 940 and a wired connection betweenthe peripheral device 940 and the computing device 920 ₃). In someembodiments, the peripheral device 940 is itself a computing device(sometimes called a “smart” device). In other embodiments, theperipheral device 940 is not a computing device (sometimes called a“dumb” device).

Depicted in FIG. 10 is a block diagram of an embodiment of a computingdevice 1000. Any of the computing devices 920 and/or any other computingdevice described herein may include some or all of the components andfeatures of the computing device 1000. In some embodiments, thecomputing device 1000 is one or more of a desktop computer, a mobilephone, a tablet, a phablet, a notebook computer, a laptop computer, adistributed system, a gaming console (e.g., an Xbox, a Play Station, aWii), a watch, a pair of glasses, a key fob, a radio frequencyidentification (RFID) tag, an ear piece, a scanner, a television, adongle, a camera, a wristband, a wearable item, a kiosk, an inputterminal, a server, a server network, a blade, a gateway, a switch, aprocessing device, a processing entity, a set-top box, a relay, arouter, a network access point, a base station, any other deviceconfigured to perform the functions, operations, and/or processesdescribed herein, or any combination thereof. Such functions,operations, and/or processes may include, for example, transmitting,receiving, operating on, processing, displaying, storing, determining,creating/generating, monitoring, evaluating, comparing, and/or similarterms used herein. In one embodiment, these functions, operations,and/or processes can be performed on data, content, information, and/orsimilar terms used herein.

In the depicted embodiment, the computing device 1000 includes aprocessing element 1005, memory 1010, a user interface 1015, and acommunications interface 1020. The processing element 1005, memory 1010,a user interface 1015, and a communications interface 1020 are capableof communicating via a communication bus 1025 by reading data fromand/or writing data to the communication bus 1025. The computing device1000 may include other components that are capable of communicating viathe communication bus 1025. In other embodiments, the computing devicedoes not include the communication bus 1025 and the components of thecomputing device 1000 are capable of communicating with each other insome other way.

The processing element 1005 (also referred to as one or more processors,processing circuitry, and/or similar terms used herein) is capable ofperforming operations on some external data source. For example, theprocessing element may perform operations on data in the memory 1010,data receives via the user interface 1015, and/or data received via thecommunications interface 1020. As will be understood, the processingelement 1005 may be embodied in a number of different ways. In someembodiments, the processing element 1005 includes one or more complexprogrammable logic devices (CPLDs), microprocessors, multi-coreprocessors, co processing entities, application-specific instruction-setprocessors (ASIPs), microcontrollers, controllers, integrated circuits,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), programmable logic arrays (PLAs), hardwareaccelerators, any other circuitry, or any combination thereof. The termcircuitry may refer to an entirely hardware embodiment or a combinationof hardware and computer program products. In some embodiments, theprocessing element 1005 is configured for a particular use or configuredto execute instructions stored in volatile or nonvolatile media orotherwise accessible to the processing element 1005. As such, whetherconfigured by hardware or computer program products, or by a combinationthereof, the processing element 1005 may be capable of performing stepsor operations when configured accordingly.

The memory 1010 in the computing device 1000 is configured to storedata, computer-executable instructions, and/or any other information. Insome embodiments, the memory 1010 includes volatile memory (alsoreferred to as volatile storage, volatile media, volatile memorycircuitry, and the like), non-volatile memory (also referred to asnon-volatile storage, non-volatile media, non-volatile memory circuitry,and the like), or some combination thereof.

In some embodiments, volatile memory includes one or more of randomaccess memory (RAM), dynamic random access memory (DRAM), static randomaccess memory (SRAM), fast page mode dynamic random access memory (FPMDRAM), extended data-out dynamic random access memory (EDO DRAM),synchronous dynamic random access memory (SDRAM), double data ratesynchronous dynamic random access memory (DDR SDRAM), double data ratetype two synchronous dynamic random access memory (DDR2 SDRAM), doubledata rate type three synchronous dynamic random access memory (DDR3SDRAM), Rambus dynamic random access memory (RDRAM), Twin Transistor RAM(TTRAM), Thyristor RAM (T-RAM), Zero-capacitor (Z-RAM), Rambus in-linememory module (RIMM), dual in-line memory module (DIMM), single in-linememory module (SIMM), video random access memory (VRAM), cache memory(including various levels), flash memory, any other memory that requirespower to store information, or any combination thereof.

In some embodiments, non-volatile memory includes one or more of harddisks, floppy disks, flexible disks, solid-state storage (SSS) (e.g., asolid state drive (SSD)), solid state cards (SSC), solid state modules(SSM), enterprise flash drives, magnetic tapes, any other non-transitorymagnetic media, compact disc read only memory (CD ROM), compactdisc-rewritable (CD-RW), digital versatile disc (DVD), Blu-ray disc(BD), any other non-transitory optical media, read-only memory (ROM),programmable read-only memory (PROM), erasable programmable read-onlymemory (EPROM), electrically erasable programmable read-only memory(EEPROM), flash memory (e.g., Serial, NAND, NOR, and/or the like),multimedia memory cards (MMC), secure digital (SD) memory cards, MemorySticks, conductive-bridging random access memory (CBRAM), phase-changerandom access memory (PRAM), ferroelectric random-access memory (FeRAM),non-volatile random access memory (NVRAM), magneto-resistive randomaccess memory (MRAM), resistive random-access memory (RRAM), SiliconOxide-Nitride-Oxide-Silicon memory (SONOS), floating junction gaterandom access memory (FJG RAM), Millipede memory, racetrack memory, anyother memory that does not require power to store information, or anycombination thereof.

In some embodiments, memory 1010 is capable of storing one or more ofdatabases, database instances, database management systems, data,applications, programs, program modules, scripts, source code, objectcode, byte code, compiled code, interpreted code, machine code,executable instructions, or any other information. The term database,database instance, database management system, and/or similar terms usedherein may refer to a collection of records or data that is stored in acomputer-readable storage medium using one or more database models, suchas a hierarchical database model, network model, relational model,entity relationship model, object model, document model, semantic model,graph model, or any other model.

The user interface 1015 of the computing device 1000 is in communicationwith one or more input or output devices that are capable of receivinginputs into and/or outputting any outputs from the computing device1000. Embodiments of input devices include a keyboard, a mouse, atouchscreen display, a touch sensitive pad, a motion input device,movement input device, an audio input, a pointing device input, ajoystick input, a keypad input, peripheral device 940, foot switch, andthe like. Embodiments of output devices include an audio output device,a video output, a display device, a motion output device, a movementoutput device, a printing device, and the like. In some embodiments, theuser interface 1015 includes hardware that is configured to communicatewith one or more input devices and/or output devices via wired and/orwireless connections.

The communications interface 1020 is capable of communicating withvarious computing devices and/or networks. In some embodiments, thecommunications interface 1020 is capable of communicating data, content,and/or any other information, that can be transmitted, received,operated on, processed, displayed, stored, and the like. Communicationvia the communications interface 1020 may be executed using a wired datatransmission protocol, such as fiber distributed data interface (FDDI),digital subscriber line (DSL), Ethernet, asynchronous transfer mode(ATM), frame relay, data over cable service interface specification(DOCSIS), or any other wired transmission protocol. Similarly,communication via the communications interface 1020 may be executedusing a wireless data transmission protocol, such as general packetradio service (GPRS), Universal Mobile Telecommunications System (UMTS),Code Division Multiple Access 2000 (CDMA2000), CDMA2000 1X (1xRTT),Wideband Code Division Multiple Access (WCDMA), Global System for MobileCommunications (GSM), Enhanced Data rates for GSM Evolution (EDGE), TimeDivision-Synchronous Code Division Multiple Access (TD-SCDMA), Long TermEvolution (LTE), Evolved Universal Terrestrial Radio Access Network(E-UTRAN), Evolution-Data Optimized (EVDO), High Speed Packet Access(HSPA), High-Speed Downlink Packet Access (HSDPA), IEEE 802.11 (WiFi),WiFi Direct, 802.16 (WiMAX), ultra wideband (UWB), infrared (IR)protocols, near field communication (NFC) protocols, Wibree, Bluetoothprotocols, wireless universal serial bus (USB) protocols, or any otherwireless protocol.

As will be appreciated by those skilled in the art, one or morecomponents of the computing device 1000 may be located remotely fromother components of the computing device 1000 components, such as in adistributed system. Furthermore, one or more of the components may becombined and additional components performing functions described hereinmay be included in the computing device 1000. Thus, the computing device1000 can be adapted to accommodate a variety of needs and circumstances.The depicted and described architectures and descriptions are providedfor exemplary purposes only and are not limiting to the variousembodiments described herein.

Embodiments described herein may be implemented in various ways,including as computer program products that comprise articles ofmanufacture. A computer program product may include a non-transitorycomputer-readable storage medium storing applications, programs, programmodules, scripts, source code, program code, object code, byte code,compiled code, interpreted code, machine code, executable instructions,and/or the like (also referred to herein as executable instructions,instructions for execution, computer program products, program code,and/or similar terms used herein interchangeably). Such non-transitorycomputer-readable storage media include all computer-readable media(including volatile and non-volatile media).

As should be appreciated, various embodiments of the embodimentsdescribed herein may also be implemented as methods, apparatus, systems,computing devices, and the like. As such, embodiments described hereinmay take the form of an apparatus, system, computing device, and thelike executing instructions stored on a computer readable storage mediumto perform certain steps or operations. Thus, embodiments describedherein may be implemented entirely in hardware, entirely in a computerprogram product, or in an embodiment that comprises combination ofcomputer program products and hardware performing certain steps oroperations.

Embodiments described herein may be made with reference to blockdiagrams and flowchart illustrations. Thus, it should be understood thatblocks of a block diagram and flowchart illustrations may be implementedin the form of a computer program product, in an entirely hardwareembodiment, in a combination of hardware and computer program products,or in apparatus, systems, computing devices, and the like carrying outinstructions, operations, or steps. Such instructions, operations, orsteps may be stored on a computer readable storage medium for executionbuy a processing element in a computing device. For example, retrieval,loading, and execution of code may be performed sequentially such thatone instruction is retrieved, loaded, and executed at a time. In someexemplary embodiments, retrieval, loading, and/or execution may beperformed in parallel such that multiple instructions are retrieved,loaded, and/or executed together. Thus, such embodiments can producespecifically configured machines performing the steps or operationsspecified in the block diagrams and flowchart illustrations.Accordingly, the block diagrams and flowchart illustrations supportvarious combinations of embodiments for performing the specifiedinstructions, operations, or steps.

For purposes of this disclosure, terminology such as “upper,” “lower,”“vertical,” “horizontal,” “inwardly,” “outwardly,” “inner,” “outer,”“front,” “rear,” and the like, should be construed as descriptive andnot limiting the scope of the claimed subject matter. Further, the useof “including,” “comprising,” or “having” and variations thereof hereinis meant to encompass the items listed thereafter and equivalentsthereof as well as additional items. Unless limited otherwise, the terms“connected,” “coupled,” and “mounted” and variations thereof herein areused broadly and encompass direct and indirect connections, couplings,and mountings. Unless stated otherwise, the terms “substantially,”“approximately,” and the like are used to mean within 5% of a targetvalue.

The principles, representative embodiments, and modes of operation ofthe present disclosure have been described in the foregoing description.However, aspects of the present disclosure which are intended to beprotected are not to be construed as limited to the particularembodiments disclosed. Further, the embodiments described herein are tobe regarded as illustrative rather than restrictive. It will beappreciated that variations and changes may be made by others, andequivalents employed, without departing from the spirit of the presentdisclosure. Accordingly, it is expressly intended that all suchvariations, changes, and equivalents fall within the spirit and scope ofthe present disclosure, as claimed.

1. A system comprising: a transportation system configured to transportvacuum seal packages, wherein each of the vacuum seal packages includesa food product; an image sensor system including an image data capturesystem, wherein the image data capture system is arranged to captureimage data of individual vacuum seal packages as the vacuum sealpackages are transported by the transportation system; and one or morecomputing devices communicatively coupled to the image sensor system andconfigured to receive the image data from the image sensor system;wherein the one or more computing devices include instructions that, inresponse to execution of the instructions by the one or more computingdevices, cause the one or more computing devices to: classify a state ofone or more of the vacuum seal packages based on the image data using atrained classification model, and output the state of the one or more ofthe vacuum seal packages after classification of the state of the one ormore of the vacuum seal packages.
 2. The system of claim 1, wherein thetrained classification model includes a decision-making processconfigured to receive an input that includes the image data and tooutput an output that includes the state of the one or more of thevacuum seal packages.
 3. The system of claim 2, wherein thedecision-making process is a multilayer neural network, wherein themultilayer neural network includes an input layer comprising the input,an output layer comprising the output, and at least one hidden layerbetween the input layer and the output layer.
 4. The system of claim 1,wherein the image sensor system further comprises a presence detectorsystem configured to detect one of the vacuum seal packages on thetransport system.
 5. The system of claim 4, wherein: the image sensorsystem further comprises a controller; the controller is configured toreceive a signal from the presence detector system indicating thedetected one of the vacuum seal packages; and the controller is furtherconfigured to control a timing of the image sensor system during atleast a portion of a time that the image sensor system obtains the imagedata of the detected one of the vacuum seal packages.
 6. The system ofclaim 5, wherein the transportation system comprises a conveyor belt,and wherein the controller is further configured to control the timingof the image sensor system based in part on a speed of the conveyorbelt.
 7. The system of claim 1, wherein the classified state of the oneor more of the vacuum seal packages includes an indication of whetherthe one or more of the vacuum seal packages is defective, isnon-defective, or has a particular defect.
 8. The system of claim 7,wherein the classified state of the one or more of the vacuum sealpackages further includes a degree of certainty as to whether the vacuumseal package is defective, is non-defective, or has a particular defect.9. The system of claim 1, wherein the one or more computing devices areconfigured to output the state of the one or more of the vacuum sealpackages via by at least one of providing an indication of the state toa user interface output device, communicating the state via acommunication interface to an external device, or storing the state in alocal database.
 10. A computer-readable medium having instructionsembodied thereon, wherein the instructions comprise instructions that,in response to execution by one or more computing devices, cause the oneor more computing devices to: receive training image data, the trainingimage data comprising image data about a plurality of first vacuum sealpackages; receive labels associated with the plurality of first vacuumseal packages, wherein each of the labels includes a state of one of theplurality of first vacuum seal packages; develop a trainedclassification model based on the training image data and the receivedlabels; receive image data representative of a second vacuum sealpackage; input the image data into the trained classification model,wherein the trained classification model is configured to classify astate of the second vacuum seal package based on the image data; andreceive the state of the second vacuum seal package from the trainedclassification model.
 11. The computer-readable medium of claim 10,wherein the state of the second vacuum seal package includes anindication of whether the second vacuum seal package is defective, isnon-defective, or has a particular defect.
 12. The computer-readablemedium of claim 11, wherein the state of the second vacuum seal packagefurther includes a degree of certainty as to whether the second vacuumseal package is defective, is non-defective, or has a particular defect.13. The computer-readable medium of claim 12, wherein the instructionsfurther comprise instructions that, in response to execution by the oneor more computing devices, further cause the one or more computingdevices to: determine, based on the degree of certainty, whether aconfidence level of the state of the second vacuum seal package is low;and in response to determining that the confidence level of the state ofthe second vacuum seal package is low, flag the second vacuum sealpackage for manual classification.
 14. The computer-readable medium ofclaim 13, wherein the instructions further comprise instructions that,in response to execution by the one or more computing devices, furthercause the one or more computing devices to: receive a user input of amanual classification of the second vacuum seal package; and furtherdevelop the trained classification model based on the image data and themanual classification of the second vacuum seal package.
 15. Thecomputer-readable medium of claim 10, wherein the trained classificationmodel includes a detection decision-making process and a classificationdecision-making process.
 16. The computer-readable medium of claim 15,wherein the detection decision-making process is configured to processthe image data to produce processed image data.
 17. Thecomputer-readable medium of claim 16, wherein the detectiondecision-making process is configured to perform at least one of:process the image data to produce processed image data at least bycropping an image in the image data so that the second vacuum sealpackage remains in the cropped image detect a presence of the secondvacuum seal package in the image data; or classify the state of thesecond vacuum seal package based on the processed image data. 18.-19.(canceled)
 20. The computer-readable medium of claim 10, wherein theinstruction that cause the one or more computing devices to develop atrained classification model include instructions that, in response toexecution by the one or more computing devices, cause the one or morecomputing devices to: train the classification model for a plurality oflearning parameters; and determine one or more model parameters based onthe plurality of learning parameters.
 21. The computer-readable mediumof claim 20, wherein the instruction that cause the one or morecomputing devices to develop a trained classification model furtherinclude instructions that, in response to execution by the one or morecomputing devices, cause the one or more computing devices to: createthe trained classification model based on the one or more modelparameters.
 22. The computer-readable medium of claim 10, wherein: theimage data representative of the second vacuum seal package includes aplurality of forms of image data the plurality of forms of image dataincludes at least two images of the second vacuum seal package; and thetrained classification model is configured to classify the state of thesecond vacuum seal package based on the image data in part by separatelyclassifying a state of each of the at least two images of the secondvacuum seal package.
 23. (canceled)